Handheld, or mobile, devices such as mobile phones, personal digital assistants (“PDAs”), audio and/or video players and other handheld devices have transformed the modern world by providing many practical bundled features within a single package. One such feature is the built in camera. A typical camera for a mobile device is capable of performing still image and video capture. However, due to the limited space and battery life available for camera equipment within mobile devices, the photographic functionality is limited when compared to conventional digital cameras.
For instance, a mobile camera typical has a short, typically 2 to 6 mm, fixed focal length with a fixed aperture (e.g., f/#=2.8). In addition, a mobile device camera has limited memory and digital signal processor (“DSP”) capabilities when compared to conventional digital cameras. Mobile device cameras are typically more inexpensive than conventional digital cameras as well.
One problem associated with all forms of handheld photography is hand shake blur. The movement of the camera due to a user hand shake during a picture capture creates a motion blur. Unlike other forms of motion blur, hand shake blur exhibits a random pattern and is difficult to predict based on past motion of the camera.
In the past, one solution that reduces the degree of hand shake blur is to capture images using shorter exposure intervals. This, however, increases the amount of noise in the image, especially in dark scenes. An alternative approach is to try to remove the blur off-line. Blur is usually modeled as a linear convolution of an image with a blurring kernel. Image deconvolution is the process of recovering the unknown image from its blurred version, given a blurring kernel. In most situations the blurring kernel is unknown as well, and the task also requires the estimation of the underlying blurring kernel. Such a process is usually referred to as blind deconvolution. However, blur caused by hand shake cannot be predicted as to direction and speed from one image to the next, which makes blind deconvolution less reliable.
Conventional digital cameras may employ techniques to reduce hand shake blur which are unavailable for mobile device cameras. One technique commonly known in the art is optical image stabilization (OIS). OIS attempts to stabilize the image by altering the optical path to the sensor within the lens itself. Typically hand shake is sensed using one or more gyroscopic sensors and a floating lens element is moved using electromagnets. Such additional components are not feasible in mobile phone cameras due to size and weight, expense and power consumption.
Another technique employed in conventional digital cameras to reduce hand shake blur is known as mechanical image stabilization where the sensor may be moved to counteract the motion of the camera. Manufacturers include components such as angle speed sensors, drive motors, and DSP chips to analyze motion and move the sensor accordingly. As with OIS technology, the size and weight, expense and power consumption of mechanical image stabilization precludes use in mobile device cameras.
Digital image stabilization is a technique where extensive image processing tries to use consecutive frames taken at a high speed, or various extrapolation techniques to reconstruct a less blurred image. However, a typical mobile device camera does not capture the appropriate frames per second to reconstruct an image adequately and the image processing required to extrapolate a less blurred image is expensive and can increase power consumption.
A need exists for hand shake image stabilization in mobile devices that may be robust, repeatable under various conditions and relatively inexpensive. The disclosed operations do not add size or weight and do not unnecessarily reduce battery life.